feat: RFC-0045 step 4, field operations on references and repeaters - #214

RFC-0045 rollout step 4: the four RFC-0041 field operations work on reference and repeater fields, with the new conversion rows. The types stay refused on every production write path until step 7. The shared suites enable them with SiteGraphStoreOptions.fieldTypes.

Closes #213 (first commit).

What changed

• RFC-0045: release purge gate-read failure with a draft that dropped reference fields sends the closure-less stored plan #213 (references/release-purge.ts): when the release gate's read of the published site node fails, it is retried once, then the run purges everything. It never falls back to the working row. "Amended in 3b" and watch item 2 are updated, and there are two new cases in references-purge.test.ts.

• sub (fields/migration-sub.ts, fields-operations/plan-sub.ts):

  • The marker sits on the repeater: { id, sub, fromKey | fromType }, or { id, sub } for a removal. A removed sub-field keeps its definition until phase C.
  • Each item goes through the sub-field's own rewriteFields rule. An item left empty is dropped, and an emptied list is no value.
  • Top-level readers ignore a sub-field rename's fromKey (renamedFrom). This covers resolveFields, rows, list values, migratedFieldValues, release deps, Studio's form model, and the rows INSERT's fromKey branch (AND migration.sub IS NULL).
  • referenceUses, definitionOfPath and fieldAssetRefs read the old sub-key while the rename runs.

• change_field_type inputs (fields-operations/retype.ts): collections, maxItems, subFields and itemLabel. The dry run checks the end state against the contract and against checkFieldDefinitions with the store's capability, which is where fields_unsupported comes from now. The field_operation_unsupported refusal is lifted.

• Conversion rows (fields/conversions-references.ts):

  • reference single ↔ multiple;
  • text ↔ reference;
  • text → repeater;
  • repeater → text;
  • multiple reference ↔ repeater.

  The maps come from references/conversion-load.ts. They are read once per dry run, phase-B pass and revert pass, and by EntryEngine.write only while such a conversion runs.

• Phase B's target guard: fresh ids only (rewriteTargetGuard mode fresh).

• set_field_localized phase A: references/localize.ts diffs the old and new site bodies in writeSite, anchored on the new site hash. The revert's first step uses the same diff.

• Revert dropped: references/revert-drops.ts, with members read once per pass and guard mode present.

• field_operation_running: unchanged from 2a. It is now tested through the real operation.

• CLI: --collections, --max-items, --sub-fields <file.json> and --sub. The admin API's operation route parses the same inputs, plus itemLabel. MCP's tool inputs are unchanged (step 5); the tools compile and do not pass the new inputs.

Deviations and decisions (in "Amended in 4")

1. Phase B guards only newly named ids. These are ids that appear nowhere in the stored value. Ids that a reshape or rename carries over stay as stored, and their rows are kept. Guarding carried ids would skip an entry whose target was deleted on every pass, so phase C would never run.
2. Writers compare against the stored entry normalized through the same markers. This is used for the target guard and for canonical ids, so a person's edit that leaves a reshaped or renamed value alone is not re-checked. A text → reference value converted on a person's write is not guarded as newly chosen. Deletes in the target collections are already refused, and only archiving can race it.
3. hasReferenceFields includes a field still converting from references. Without this, reference → text left stale rows on sites with no other reference field. The suite caught it.
4. Repeater sources need exactly one sub-field. Readers only see the new definition plus the marker, so the repeater → text and repeater → reference rows read each item's single value from its shape. A rich-text sub-field joins as its Markdown source. Repeater → text sets multiline: true.
5. Id-carrying conversions may only widen collections. A reference → reference retype is allowed only to switch multiple.
6. Text → reference skips the entry's own group (ConvertContext.group).
7. Phase C's pending check tells a list of ids from a list of items by the first element's JSON type, because both are arrays.
8. R16 under a running operation resolves by retry. In a site with reference fields every entry write carries the definitions anchor. A holder write planned before phase A therefore fails its batch, and the retry reads the new shared/fromKey. The tests assert exactly that.
9. Watch item 3: live referencedBy lists a converting referrer only after phase B, as documented. The live-versus-snapshot comparison runs without a marker, so nothing breaks.

Tests

• testing-field-operations-references.ts (new, 14 cases), run on SQLite, Miniflare D1 and DO SQLite. It covers:
  • every new conversion row end to end, with rows;
  • sub rename, convert and remove, with rows and asset rows following;
  • a target archived or deleted during phase B;
  • resume;
  • revert with a deleted target (dropped);
  • set_field_localized phase A's shared in both directions, with a racing holder write;
  • rename with fromKey;
  • R12 through the real operation, including a delete planned before phase A and batched after it.
• fields-conversions-references.test.ts (pure): every row, plus the sub markers in rewriteFields, normalizeThroughMarkers and resolveFields.
• fields-cli.test.ts: the new flags, and --sub-fields refused by the local store.
• The gate and references suites are updated for the lifted refusal.

Size: about 2.3k added lines, of which about 1k are tests. Splitting the CLI and revert dropped into a 4b would move only about 300 lines, so this stays one PR. I can split it if you prefer.

pnpm check:

• Lint, the size ratchet, the generated-file checks, typecheck, and every package's tests pass.
• Two failures came up under load during the run:
  • @cloudhal/ui's analytics.test.tsx (a date-picker in a package this PR does not touch) failed once and passes alone, 31/31.
  • @cloudhal/api's fields-api test still expected the route to refuse reference targets. It is now updated to expect 409 fields_unsupported from the store's dry run, and the full API suite passes alone, 641/641.
• Turbo stopped the rest of the run after the UI failure. I re-ran the remaining test tasks with --continue, and they all passed except that API test.

🤖 Generated with Claude Code